Click Here!
home account info subscribe login search My ITKnowledge FAQ/help site map contact us


 
Brief Full
 Advanced
      Search
 Search Tips
To access the contents, click the chapter and section titles.

Oracle Performance Tuning and Optimization
(Publisher: Macmillan Computer Publishing)
Author(s): Edward Whalen
ISBN: 067230886x
Publication Date: 04/01/96

Bookmark It

Search this book:
 
Previous Table of Contents Next


Results

The ultimate outcome of running a TPC benchmark is to publish the performance of the system. Other results are published as well. For a TPC benchmark to be valid, a Full Disclosure Report (FDR) must be submitted. The FDR includes the Executive Summary, which has highlights of the FDR and usually includes the following information:

  The benchmark sponsor(s). These are the companies that actually did the work for the benchmark. Typically, the primary sponsor is the hardware vendor and may additionally include the OS and RDBMS vendors.
  The system performance and price/performance. These are the primary metrics used to compare system performance.
  The system configuration. This includes the system model number, number and type of CPUs, number of controllers and disk drives, network configuration, and so on.
  A price list of all the components in the system. This is a line-item list of all the components used, including part numbers and prices. The list also includes the support costs, which are required to be disclosed.
  The response times of specific transactions. Depending on the benchmark, this may include keystroke times, transaction response times, and think times.
  Transaction mix. Depending on the benchmark, the mix that was achieved is disclosed.
  The configuration size. Depending on the benchmark, this can be the number of users simulated, the number of warehouses used, and so on.

As you can see, the Executive Summary is valuable in providing a quick reference to the benchmark and the performance of the system. The Executive Summary is typically two to three pages in length and is available electronically from the TPC Web site (at this URL: http://www.tpc.org/).

In addition to the Executive Summary, the FDR includes the following information about the benchmark:

  Configuration details. Differences between the simulated configuration and the tested configuration are disclosed.
  Response time graphs. Graphs of the response times for each transaction type are provided.
  Attestation letter. A letter from the independent auditor attesting to the fact that the benchmark met all the rules of the specification.
  Price quotations. In some cases, a price quotation must be provided to verify the cost of the system.
  Tuning parameters. All OS and RDBMS tuning parameters must be disclosed. This information allows the benchmark to be reproduced by a competitor to verify the performance results.
  Source code. All application code must be disclosed. A rule was passed in the TPC in 1994 that allows all source code published in a benchmark to be considered public domain software within the TPC. This allows other vendors to copy the application code to be used in their benchmarks.

The FDR is quite detailed and provides a great deal of information about the test that was run. Usually, the vendor that ran the benchmark is happy to send you a copy of the FDR if you are interested.

Of course, the content of the FDRs and Executive Summaries varies depending on the benchmark that was run. The following section looks at the benchmarks that have come out of the TPC.

Benchmarks

This section describes what is involved in publishing a TPC benchmark. To publish a benchmark, the following steps are typically involved:

1.  TPC application. The TPC benchmarks are only specifications for a workload. Because of the diversity of OS and RDBMS products, it is necessary to develop specific code for your particular RDBMS and OS. This code must be completely compliant with the workload defined in the specification. It is up to the vendor running the test to develop this application.
2.  Tune. Once you have working benchmark code, the tuning stage begins. It can take days, weeks, or even months to achieve optimal performance. There may be additional tuning involved to meet the response time criteria or other part of the specification.
3.  Simulate. You must either design and build your own keystroke-generator product or purchase one of the commercially available products. In several of the TPC benchmarks, you must actually press keys for the application and verify that the proper responses are sent back from the application.
4.  Audit. An independent auditor must come on-site at your expense to verify compliance with the specification. The auditor also verifies that the system passes all the ACID tests. The audit can take several days or weeks.
5.  Full Disclosure Report (FDR). A full disclosure report must be written to specify the performance rates achieved, system cost, and adherence to specification; the report must also include all source code used in your application.
6.  Marketing. Only after the FDR has reached the TPC can you announce the TPC benchmark results you have achieved.

As you can imagine, TPC benchmarks can be quite time consuming and expensive. It is not only expensive to audit and run the test but the amount of hardware necessary to achieve the results can be quite large.

Although the benchmarks are expensive and difficult to run, they are seen as a great marketing tool. Each year since the TPC was founded, there has been an increase in the publication of benchmark results. As more and more benchmarks become available, I anticipate that the publication of results will continue to increase.

The year 1995 marked a milestone for the TPC with the retirement of its first two benchmarks: the TPC-A and TPC-B. The TPC is left with five active benchmarks: TPC-C, TPC-D, TPC-E, TPC-C/S, and TPC-Server benchmarks, each of which is described in the following sections.

Although the TPC-A and TPC-B benchmarks are now obsolete, you may have results from these benchmarks or may hear them mentioned. Because they are still referred to in the industry, they are included in the discussion of the TPC benchmarks that follows. Even though the TPC-A and TPC-B benchmarks have been officially retired and no more benchmarks can be published, these benchmarks are still used extensively within the computer industry. The TPC-B benchmark is an excellent load test that stresses several vital subsystems.


Previous Table of Contents Next


Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc.
All rights reserved. Reproduction whole or in part in any form or medium without express written permission of EarthWeb is prohibited.